Swallowing estimation device, information terminal device, and storage medium

ABSTRACT

A swallowing estimation device includes: a sound detection part configured to detect sound of a larynx portion; a respiration detection part configured to detect respiration; and a swallowing estimation part configured to estimate swallowing based on sound information outputted from the sound detection part and based on respiration information outputted from the respiration detection part. The swallowing estimation part obtains a value of a parameter for swallowing estimation with respect to a biological sound generation interval that corresponds to a respiratory cessation interval longer than or equal to 400 msec, and estimates whether swallowing has occurred in the biological sound generation interval based on whether the obtained value of the parameter satisfies a swallowing determination condition.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of International Application No.PCT/JP2014/62239 filed May 7, 2014, which claims priority to JapanesePatent Application No. 2013-174949 filed Aug. 26, 2013. The contents ofthese applications are incorporated herein by reference in theirentirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a swallowing estimation device whichestimates a swallowing movement, an information terminal device forobtaining information necessary for estimation of the swallowingmovement, and a storage medium having stored therein a program whichprovides a computer with a swallowing estimation function.

2. Disclosure of Related Art

Japanese Laid-Open Patent Publication No. 2013-17694 discloses atechnology which analyzes the frequency of biological sound obtainedfrom a larynx portion, and distinguishes among swallowing, coughs andutterance based on frequency characteristics thereof. Meanwhile,“Non-restrictive monitoring of swallowing frequency of elderlyindividuals” in an academic journal “The Journal of JapaneseOccupational Therapy Association” Vol. 31(1) PP. 52 to 59 (Non-patentliterature) issued by the Japanese Association of OccupationalTherapists in February in 2012 discloses the following technology. Thatis, with respect to pulse groups each consisting of pulses obtained byconverting biological sound collected by a larynx microphone intopulses, a pulse group in which each pulse width is less than or equal to60 msec and the number of pulses is less than or equal to 20 isdetermined as representing a swallowing movement.

Before performing swallowing estimation, first, it is necessary tocollect information such as biological sound from a subject. In thiscase, in order not to restrict the subject as much as possible, it isdesirable that the apparatus and the like to be worn by the subject areas simple as possible. Accordingly, it becomes possible to collectinformation for swallowing estimation from the subject in a livingenvironment. However, when information is collected in a livingenvironment in this manner, information of various sounds thatcontaminate swallowing sound is collected, such as household noises,sounds during eating and drinking, conversations, sounds generated whenthe neck is rotated, in addition to biological sound generated duringswallowing. These noises cause erroneous swallowing estimation,resulting in reduced estimation accuracy of swallowing.

SUMMARY OF THE INVENTION

During swallowing, there is always a respiratory cessation (apnea) statefor a predetermined period or longer. When this is focused on, it isdesirable that a swallowing interval is estimated by performing analysisof a signal from an organism only with respect to an apneic intervalthat is longer than or equal to a predetermined period.

A first aspect of the present invention relates to a swallowingestimation device. The swallowing estimation device according to thisaspect includes: a sound detection part configured to detect sound of alarynx portion; a respiration detection part configured to detectrespiration; and a swallowing estimation part configured to estimateswallowing based on sound information outputted from the sound detectionpart and based on respiration information outputted from the respirationdetection part. Here, the swallowing estimation part obtains a value ofa parameter for swallowing estimation with respect to a biological soundgeneration interval that corresponds to an apneic interval longer thanor equal to 400 msec, and estimates whether swallowing has occurred inthe biological sound generation interval based on whether the obtainedvalue of the parameter satisfies a swallowing determination condition.

A second aspect of the present invention relates to a swallowingestimation device. The swallowing estimation device according to thisaspect includes: a biological sound detection means configured to detectbiological sound at a larynx portion; respiration detection meansconfigured to detect change in airflow of respiration; a signalintensity conversion means configured to convert biological sound dataobtained by sampling the biological sound into signal intensity data;signal interval identification means configured to identify a signalinterval having an intensity level higher than or equal to a noise levelbased on the signal intensity data; respiration identification meansconfigured to identify an apneic interval based on airflow pressure dataobtained by sampling change in the respiration; signal pulsing meansconfigured to obtain a signal intensity that corresponds to a samplingtiming in the apneic interval that is longer than or equal to apredetermined period and that overlaps the signal interval, andconfigured to generate a signal pulse having a width that corresponds toa period in which the signal intensity is greater than or equal to apredetermined level; swallowing reflex estimation means configured toestimate, as an estimated swallowing reflex interval, the apneicinterval that satisfies a determination condition that the number of thesignal pulses in the apneic interval longer than or equal to thepredetermined period is less than or equal to a predetermined number anda width of each signal pulse in the apneic interval longer than or equalto the predetermined period is less than or equal to a predeterminedperiod; and display means configured to display the estimated swallowingreflex interval.

A third aspect of the present invention relates to an informationterminal device. The information terminal device according to thisaspect includes: a sound detection part configured to detect sound of alarynx portion; a respiration detection part configured to detectrespiration; and a storage part in which sound information outputtedfrom the sound detection part and respiration information outputted fromthe respiration detection part are stored.

A fourth aspect of the present invention is a storage medium havingstored therein a program which provides a computer with: a function ofobtaining a value of a parameter for swallowing estimation, with respectto a biological sound generation interval that corresponds to an apneicinterval longer than or equal to a predetermined period; and a functionof estimating whether swallowing has occurred in the biological soundgeneration interval based on whether the obtained value of the parametersatisfies a swallowing determination condition.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and new features of the present inventionwill be fully clarified by the following description of the embodiment,when read in conjunction with accompanying drawings.

FIG. 1 is a functional block diagram showing a configuration and afunction of a swallowing activity monitoring device according toEmbodiment 1;

FIG. 2 shows a display example of each waveform in an estimatedswallowing reflex interval obtained by the swallowing activitymonitoring device according to Embodiment 1;

FIG. 3 shows a display example of a biological sound waveform and anairflow sound waveform shown on a fine time scale in a superposed andzoomed-in manner in the swallowing activity monitoring device accordingto Embodiment 1;

FIG. 4 is an external view showing a configuration of a swallowingestimation system according to Embodiment 2;

FIG. 5 is a block diagram showing a configuration of the swallowingestimation system according to Embodiment 2;

FIG. 6 is a flow chart showing operations performed by a terminal deviceand an information processing device according to Embodiment 2;

FIG. 7 is a flow chart showing operation performed by the informationprocessing device according to Embodiment 2;

FIG. 8 is a flow chart showing operation performed by the informationprocessing device according to Embodiment 2;

FIG. 9A shows a biological sound data according to Embodiment 2;

FIG. 9B shows an airflow pressure data according to Embodiment 2;

FIG. 9C shows a hyoid bone displacement data according to Embodiment 2;

FIG. 10A schematically shows a spectrogram according to Embodiment 2;

FIG. 10B schematically shows a mel-frequency spectrogram according toEmbodiment 2;

FIG. 100 shows pulses obtained through continuous wavelet transformaccording to Embodiment 2;

FIG. 10D schematically shows pulses obtained through continuous wavelettransform in a zoomed-in manner according to Embodiment 2;

FIG. 11 shows a screen to be displayed on a display part according toEmbodiment 2;

FIG. 12 shows a screen to be displayed on the display part according toEmbodiment 2;

FIG. 13A shows a screen to be displayed on the display part according toEmbodiment 2;

FIG. 13B is a flow chart showing operation performed by the informationprocessing device according to Modification 1;

FIG. 13C shows estimation results obtained when swallowing estimationwas performed by actually collecting information for swallowingestimation;

FIG. 14 is a flow chart showing operation performed by the informationprocessing device according to Modification 2; and

FIG. 15 is a diagram for explaining the procedure of determining whetheraspiration has occurred according to Modification 3.

It should be noted that the drawings are solely for description and donot limit the scope of the present invention by any degree.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The present embodiment is characterized in that: an estimated swallowingreflex interval is constrained based on energy distribution ofbiological sound obtained from a larynx portion and based on informationof change in airflow pressure caused by respiration; and the estimatedswallowing reflex interval is further constrained to be specified basedon information of displacement of the hyoid bone.

Embodiment 1

FIG. 1 shows a circuit block diagram for explaining operation performedin Embodiment. However, the blocks in a personal computer are not actualcircuit blocks but functional blocks.

Hereinafter, operation performed during a data measurement period of thepresent embodiment will be described.

In the present embodiment, a microphone disposed close to the larynxportion functions as biological sound detection means 1 which detectssound in an organism, an air pressure sensor provided in a tube of anasal cannula functions as respiration detection means 2 which detectschange in airflow caused by respiration, and a piezoelectric sheetattached to the larynx portion functions as hyoid bone displacementdetection means 3 which catches deformation of the larynx portion todetect displacement of the hyoid bone.

A biological sound signal obtained from the biological sound detectionmeans 1, an airflow pressure signal obtained from the respirationdetection means 2, and a hyoid bone displacement signal obtained fromthe hyoid bone displacement detection means 3 are inputted to theircorresponding first sampling circuit 4, second sampling circuit 5, andthird sampling circuit 6, respectively. Then, each signal is sampled ata cycle of 0.1 msec (10 kHz), and then subjected to A/D conversion.Then, biological sound data, airflow pressure data, and hyoid bonedisplacement data which have been obtained through the A/D conversionare stored in a storage medium 8 along with time data obtained fromclock means 7. Accordingly, during a data measurement period, thebiological sound data, the airflow pressure data, and the hyoid bonedisplacement data are stored in the storage medium 8, each paired withtime data.

After the measurement period described above has ended, data processingas described below is performed in the personal computer.

First, signal intensity conversion means 9 converts, into signalintensity data, biological sound data which corresponds to a signalwaveform read from the storage medium 8. Specifically, with the windowfunction (sampling range) set to 1.5 seconds, biological sound data iscut out to be subjected to short-time Fourier transform processing, andthen, the total sum of the amplitudes is obtained, whereby thebiological sound data is converted into signal intensity data. At thistime, an appropriate bandpass filter process may be performed.

Since this short-time Fourier transform processing is performed with thesampling range shifted by 0.2 seconds, signal intensity data having atemporal resolution of 1/2000 of that of the biological sound data isoutputted. This signal intensity data is used to exclude soundlessintervals. By setting the window function to be wide and also settingthe shift interval to be large, load on the calculation processing isreduced.

Signal interval identification means 10 compares the signal intensitydata with a first reference value L1, to generate an identificationoutput for identifying a signal interval having an intensity levelhigher than or equal to a noise level. That is, the signal intervalidentification means considers that swallowing has not occurred in alow-level interval where only noise is present and that swallowing hasoccurred in a high-level interval, thereby specifying the high-levelinterval as a signal interval.

Respiration identification means 11 issues respiration identificationoutputs which respectively represent three types of intervals of airflowpressure data, i.e., expiration interval, respiratory pause (lateexpiration) interval, and inspiration interval.

Signal pulsing means 12 extracts biological sound data that correspondsto an apneic interval greater than or equal to a second reference valueL2 (for example, 0.6 seconds) in the signal interval, and performscontinuous wavelet transform processing on the extracted biologicalsound data to obtain the total sum of frequency spectrum intensities.Then, the signal pulsing means 12 compares the obtained total sum with athird reference value L3, and then, outputs a signal pulse having awidth that corresponds to a period having an intensity value greaterthan or equal to the reference value.

When the width of each of all the signal pulses in the apneic intervalgreater than or equal to the second reference value L2 in the signalinterval is less than or equal to a fourth reference value L4 (forexample, 25 msec) and the number of the signal pulses is less than orequal to a fifth reference value L5 (for example, 20), signal pulseevaluation means 13 estimates that there is a high possibility that thesignal interval is a swallowing reflex interval, and issues a firstestimation output. In the non-patent literature mentioned above, thesignal pulse width during swallowing is set to be less than or equal to60 msec. However, in the present embodiment, since the third referencevalue L3 is set to a large value, the signal pulse width is small, andthe fourth reference value L4 is set to 25 msec.

Next, bandpass evaluation means 14 converts the biological sound datainto a mel-frequency spectrogram in the apneic interval greater than orequal to the second reference value L2 in the signal interval. Then,with respect to the mel-frequency spectrogram, when the proportion ofthe total sum of spectrum levels of high range components (for example,higher than or equal to 750 Hz) in the apneic interval to the total sumof spectrum levels in the apneic interval is greater than or equal to asixth reference value L6 (for example, 15%), the bandpass evaluationmeans 14 estimates that there is a high possibility that the signalinterval is a swallowing reflex interval, and issues a second estimationoutput.

Further, hyoid bone displacement evaluation means 15 specifies themaximum value of the hyoid bone displacement data in the measurementperiod. Then, when hyoid bone displacement data whose proportion to themaximum value is greater than or equal to a seventh reference value L7(for example, 10%) is present in the apneic interval greater than orequal to the second reference value L2 in the signal interval, the hyoidbone displacement evaluation means 15 estimates that there is a highpossibility that the signal interval is a swallowing reflex interval,and issues a third estimation output.

In the present embodiment, swallowing reflex estimation means 16generates an output that specifies the signal interval as an estimatedswallowing reflex interval, only when all of the first estimationoutput, the second estimation output, and the third estimation outputhave been generated.

The estimated swallowing reflex interval specified as above is displayedvia display control means 17 as shown in FIGS. 2A to 2D on display means18 which is a monitor screen of the personal computer. On the displaymeans 18, in accordance with a time scale indicated in a reduced manner,a biological sound signal waveform in FIG. 2A, an airflow pressuresignal waveform in FIG. 2B, a hyoid bone displacement signal waveform inFIG. 2C, and a biological sound signal intensity waveform in FIG. 2D ofthe entire measurement period are displayed as a basic screen along withthe time scale.

In FIG. 2A, each vertical broken line indicates an estimated swallowingreflex position. With respect to the airflow pressure waveform shown inFIG. 2B, in an actual display, the respiration identification outputswhich respectively represent three types of intervals of airflowpressure data, i.e., expiration interval, respiratory pause interval,and inspiration interval, are colored differently and indicated withvertical lines, so that the respective types of outputs can berecognized by viewing.

In this state, when an operator points at an estimated swallowing reflexposition in the basic screen, the biological sound signal waveform, theairflow pressure signal waveform, and the hyoid bone displacement signalwaveform of an apneic interval corresponding to the pointed position andseveral respiration periods before and after the apneic interval aredisplayed as a sub-screen, along with the time scale.

The estimated swallowing reflex interval above is displayed in variousmanners in order to indicate a swallowing state of aspiration or a highaspiration risk.

For example, after normal swallowing, the expiratory sound intensity isless than or equal to a half of the inspiratory sound intensity, butafter abnormal swallowing, the expiratory sound intensity is increased.With this focused on, a sub-screen is displayed as explained below.

Respiratory sound intensity evaluation means 19 performs short-timeFourier transform individually on biological sound data for each ofeight intervals in total, i.e., four cycles of the expiration intervalsand the inspiration intervals before and after the estimated swallowingreflex interval, and obtains the total sum for each interval as signalintensity data. Further, when the average signal intensity of theexpiration intervals/the average signal intensity of the inspirationintervals after the estimated swallowing reflex interval has increasedto exceed 50% of that before the estimated swallowing reflex interval,the respiratory sound intensity evaluation means 19 determines theinterval as an estimated aspiration interval, and inputs alertinformation to the display control means 17. When the operator selectsthe estimated aspiration interval, the display control means 17 causesthe display means 18 to display a sub-screen for the estimatedaspiration interval instead of the sub-screen for the estimatedswallowing reflex interval described above.

When diagnosing swallowing, the respiration states before and after anestimated swallowing reflex interval can be used as a reference. Withthis focused on, the display control means 17 determines whether each ofthe respirations before and after an estimated swallowing reflexinterval is an expiration interval or an inspiration interval, expressesthe result in four types, and additionally displays the frequencythereof in the basic screen on the display means 18, as shown in FIG. 3,for example.

The display example shown in FIG. 3 indicates the following: swallowingoccurred in expiration (SWtype: E-SW); respiration after the swallowingstarted with expiration (SWtype2: SW-E); when the average respirationinterval is assumed as 1, the swallowing start point is the time pointwhen 1.04 has elapsed from the inspiration start (old phase: 1.04); andlatency before inspiration starts after the swallowing is 0.78 seconds(insp-start: 0.78). The broken vertical lines in FIG. 3 each representsa start position of inspiration, and the (interval between) dash-dotvertical lines represents a deglutition apnea period.

The indication of the estimated swallowing reflex interval in thepresent embodiment is not limited to the manner described above, andincludes various types of indication modes relevant to the estimatedswallowing reflex interval. In particular, indication of aspiration riskestimation is important among indications of swallowing reflexestimation.

In the present embodiment, the swallowing reflex estimation means 16estimates the estimated swallowing reflex interval, based on all of thefirst estimation output, the second estimation output, and the thirdestimation output. However, the estimation in the present embodiment maybe made only based on the first estimation output, or may be made basedon the first estimation output and the second estimation output.

In addition, the reference values used in the present embodiment are foradjusting the degree of constraining the estimated swallowing reflexinterval, are not intended to limit the values and ranges in the presentembodiment, and should be adjusted as appropriate.

Embodiment 2

Embodiment 2 is a more specific example of the configuration and theprocesses according to Embodiment 1.

In Embodiment 2, a swallowing estimation system 100 corresponds to the“swallowing estimation device” set forth in claims. A sound sensor 231 acorresponds to the “sound detection part” set forth in claims. Apressure sensor 215 corresponds to the “respiration detection part” setforth in claims. A control part 313 corresponds to the “swallowingestimation part” set forth in claims. A displacement sensor 231 bcorresponds to the “displacement detection part” set forth in claims. Aspeaker 312, the control part 313, and a display part 320 correspond tothe “output part” set forth in claims. A hard disk 314 corresponds tothe “storage part” set forth in claims. An input part 330 corresponds tothe “input part” set forth in claims. A terminal device 210 correspondsto the “information terminal device” set forth in claims. However, thecorrespondence between the present embodiment and the claims is merelyone example and does not limit the invention according to the claims tothe present embodiment.

FIG. 4 is an external view showing a configuration of the swallowingestimation system 100 according to the present embodiment. Theswallowing estimation system 100 includes a measurement device 200 andan information processing device 300. In the swallowing estimationsystem 100, a small storage medium 101 (for example, SD card) which iseasy to be carried is used. The measurement device 200 includes theterminal device 210, a nasal cannula 220, and a detection part 230.

The terminal device 210 includes a display part 211 and an input part212, and is configured to be small and light in weight so that a subjectcan always wear the terminal device 210. While confirming the display ofthe display part 211, the subject inputs an instruction to a controlpart 214 (see FIG. 5) through an input part 212 which includes buttonsand adjustment knobs. The terminal device 210 includes a writing part213 which performs writing on a storage medium 101.

The nasal cannula 220 includes an attachment part 221 having a pair oftube-like members, and tubes 222 connected to opposite ends of theattachment part 221. The pair of tube-like members of the attachmentpart 221 are inserted into the nasal cavities of a patient, and theother ends of the tubes 222 are connected to the terminal device 210.Accordingly, when a patient breathes, air in the tubes 222 flows, andthe flow of the air in the tubes 222 is detected as a pressure by thepressure sensor 215 (see FIG. 5) in the terminal device 210. Even whenthe patient is breathing through the mouth, since the nasal cavities areconnected to the oral cavity, the air in the tubes 222 flows and thepressure changes.

The detection part 230 includes a pad 231 which is thin and flexible,and a cable 232. The pad 231 is attached to the larynx portion of thesubject. The pad 231 includes: the sound sensor 231 a (see FIG. 5) fordetecting sound of the larynx portion; and the displacement sensor 231 b(FIG. 5) for detecting, based on the pressure, displacement of the hyoidbone depending on deformation of the larynx portion.

The information processing device 300 includes a body 310, the displaypart 320, and the input part 330. The body 310 includes a reading part311 which performs reading from the storage medium 101, and the speaker312 for outputting sound. The operator inputs an instruction to thecontrol part 313 (see FIG. 5) by using the input part 330 which includesa keyboard and a mouse. The display part 320 is composed of a display,and displays a swallowing estimation result and the like describedlater.

FIG. 5 is a block diagram showing a configuration of the swallowingestimation system 100.

The terminal device 210 includes the control part 214, the pressuresensor 215, and an A/D conversion part 216, in addition to the displaypart 211, the input part 212, and the writing part 213 which are shownin FIG. 4.

The pressure sensor 215 detects, as a pressure, the flow of the airguided via the tubes 222 of the nasal cannula 220, and outputs thedetected analog pressure signal to the A/D conversion part 216. Thedetection part 230 includes the sound sensor 231 a and the displacementsensor 231 b. The sound sensor 231 a detects sound in the vicinity ofthe larynx portion of the subject, and outputs the detected analog soundsignal to the A/D conversion part 216. The displacement sensor 231 bdetects deformation of the larynx portion of the subject as displacementof the hyoid bone, and outputs the detected analog displacement signalto the A/D conversion part 216. The A/D conversion part 216 samples thepressure signal, the sound signal, and the displacement signal inpredetermined cycles, and outputs digital signals corresponding to therespective sampled signals to the control part 214. The respectivepieces of data obtained by performing A/D conversion on the soundsignal, the pressure signal, and the displacement signal correspond tothe “biological sound data”, the “airflow pressure data”, the “hyoidbone displacement data” in Embodiment 1, respectively.

The control part 214 controls the components of the terminal device 210.Moreover, the control part 214 writes, along with time data, each dataoutputted from the A/D conversion part 216 into the storage medium 101set in the writing part 213. The time data is counted by a clock circuitbuilt in the control part 214. When measurement by the measurementdevice 200 has ended, the storage medium 101 is taken out of the writingpart 213, and is set in the reading part 311 of the informationprocessing device 300.

The body 310 is composed of a personal computer, for example, andincludes the control part 313 and the hard disk 314, in addition to thereading part 311 and the speaker 312 which are shown in FIG. 4. Thecontrol part 313 controls the components of the body 310, receives aninstruction inputted via the input part 330, outputs an image signal tothe display part 320 in response to the instruction, and outputs soundfrom the speaker 312. Moreover, the control part 313 reads data from thestorage medium 101 set in the reading part 311, and stores the data inthe hard disk 314. Further, the control part 313 performs calculationbased on the data and a program stored in the hard disk 314. The programstored in the hard disk 314 provides the control part 313 with aswallowing estimation function described later. This program may bepreviously installed in the hard disk 314, or may be downloaded to thehard disk 314 from a disk medium or the Internet. In this case, thereading part 311 is provided with a disc drive to read this program fromthe disc.

FIG. 6 to FIG. 8 are flow charts showing operations performed by theterminal device 210 and the information processing device 300.

With reference to FIG. 6, upon receiving a start instruction via theinput part 212 (S101: YES), the control part 214 of the terminal device210 obtains biological sound data, airflow pressure data, and hyoid bonedisplacement data, and starts a process of writing the obtained data inthe storage medium 101 (S102). Then, upon receiving a stop instructionvia the input part 212 (S103: YES), the control part 214 ends thewriting process (S104). Accordingly, the process of the terminal device210 ends. The storage medium 101 in which the data has been written istransferred to the information processing device 300 as described above.

FIGS. 9A to 9C respectively show the biological sound data, the airflowpressure data, and the hyoid bone displacement data written in thestorage medium 101, for a predetermined period, as the waveforms ofanalog signals before being subjected to A/D conversion. In FIGS. 9A to9C, signals in 2 seconds are extracted and shown. However, actually,data corresponding to a period for which the writing process has beenperformed is stored in the storage medium 101. “Biological soundgeneration interval” shown in FIGS. 9A and 9C and “apneic interval”shown in FIGS. 9B and 9C will be described later.

With reference back to FIG. 6, upon receiving a start instruction viathe input part 330 (S201: YES) after having stored the biological sounddata, the airflow pressure data, and the hyoid bone displacement datainto the hard disk 314, the control part 313 of the informationprocessing device 300 performs the process below.

The control part 313 creates a spectrogram by performing short-timeFourier transform on the biological sound data, and extracts biologicalsound generation intervals based on the created spectrogram (S202).Specifically, with respect to the biological sound data of the entireinterval, the control part 313 sets the window function (sampling range)to 1.5 seconds to cut out biological sound data, and performs short-timeFourier transform on the cut-out biological sound data to create aspectrogram as shown in FIG. 10A. That is, Fourier transform isperformed in a unit time (time width of 1.5 seconds), and this issequentially performed with 0.2 seconds shifted every time, whereby aspectrogram is created. The example shown in FIG. 10A is a spectrogramcreated for 20 unit-time widths, that is, for 4 seconds. Then, thecontrol part 313 obtains the total sum of the amplitudes of the createdspectrogram to perform conversion into signal intensity data, andextracts, as a biological sound generation interval, each interval thathas a value exceeding the noise average+2SD (standard deviation).Accordingly, with respect to the biological sound data of the entireinterval, biological sound generation intervals are specified. FIGS. 9Aand 9C additionally show a biological sound generation intervalextracted in this manner.

Next, with respect to the airflow pressure data, the control part 313extracts, as an apneic interval, each interval that has a value lessthan or equal to a threshold that is set in consideration of noise(S203). Accordingly, with respect to the airflow pressure data of theentire interval, apneic intervals are set. FIG. 9B additionally shows anapneic interval extracted in this manner.

Next, in each biological sound generation interval, the control part 313creates a mel-frequency spectrogram as shown in FIG. 10B, from thespectrogram created in S202 (S204). In FIG. 10B, the vertical axis isexpressed in the mel-scale. Thus, in the mel-frequency spectrogram shownin FIG. 10B, compared with the frequency spectrogram shown in FIG. 10A,the coordinate axis in low frequency bands is extended, and thecoordinate axis in high frequency bands is compressed. Accordingly, theresolving power for 0.7-5.0 kHz bands is enhanced.

Next, in each biological sound generation interval, the control part 313performs continuous wavelet transform on the data obtained in S202 thathas been subjected to the short-time Fourier transform, to convert thedata into pulses (S205), thereby to generate the pulses as shown in FIG.100. In the example shown in FIG. 100, the biological sound generationinterval includes six pulses. In an enlarged schematic representation ofthese pulses, a plurality of pulses respectively having different widthsare included as shown in FIG. 10D.

Next, from among the biological sound generation intervals extracted inS202, the control part 313 extracts each biological sound generationinterval that satisfies all of the following three conditions (S206).

The first condition is that the biological sound generation intervalincludes an amplitude of the hyoid bone displacement data whoseproportion to the maximum amplitude thereof in the entire interval isgreater than or equal to a predetermined proportion (for example, 3%).For example, in the example shown in FIG. 9C, an amplitude A1 of thehyoid bone displacement data in the biological sound generation intervalis large. When the amplitude of the hyoid bone displacement data islarge in the biological sound generation interval, the first conditionis satisfied. During swallowing, the hyoid bone goes up, then, isdisplaced forward, and then, returns to its original position. The firstcondition is for determining, based on the hyoid bone displacement data,whether such a phenomenon has occurred in the biological soundgeneration interval.

The second condition is that, in the mel-frequency spectrogram of thebiological sound generation interval, the proportion of the total sum(power) of the spectrum that is higher than or equal to 750 Hz isgreater than or equal to a predetermined proportion (for example, 15%).Normally, swallowing sound contains high frequency (>750 Hz) components.The second condition is for determining, based on the biological sounddata, whether the frequency of sound corresponding to swallowing soundhas occurred in the biological sound generation interval. For example,with respect to the example shown in FIG. 10B, in the mel-frequencyspectrogram of the biological sound generation interval, if theproportion of the total sum of the spectrum that is higher than or equalto 750 Hz exceeds 15%, the second condition is satisfied. It should benoted that the threshold is set to 750 Hz here, but this threshold canbe changed to another frequency as appropriate, by taking statistics ofactually measured values of swallowing sound.

The third condition is that the number of pulses generated in S205 inthe biological sound generation interval is less than or equal to apredetermined number (for example, 50) and that the maximum width of thepulses generated in S205 in the biological sound generation interval isless than or equal to a predetermined value (for example, 15 msec). Thisis because swallowing sound can be distinguished from other sounds fromthe viewpoint of intermittency and continuity. The higher theintermittency is, the more pulses having short widths appear, and thehigher the continuity is, the fewer pulses appear and the longer thepulse width becomes. The third condition is for determining, based onthe biological sound data, whether intermittency and continuity of soundcorresponding to swallowing sound have occurred in the biological soundgeneration interval. For example, with respect to the example shown inFIGS. 10C and 10D, in the biological sound generation interval, if thenumber of pulses N is less than or equal to 50, and the maximum pulsewidth W is less than or equal to 15 msec, the third condition issatisfied. It should be noted that the threshold for the number ofpulses is set to 50, and the threshold for the maximum pulse width isset to 15 msec here, but the threshold for the number of pulses and thethreshold for the maximum pulse width can be changed to another numberand another time width as appropriate, by taking statistics of actuallymeasured values of swallowing sound.

With reference to FIG. 7, next, the control part 313 sequentially refersto the biological sound generation intervals extracted in S206, andextracts intervals in which it can be estimated that swallowing soundhas been generated, as described below.

First, the control part 313 sets, as a reference destination, the firstbiological sound generation interval among the biological soundgeneration intervals extracted in S206 (S207). Subsequently, the controlpart 313 sets a reference range sufficiently wider than this biologicalsound generation interval for the airflow pressure data, and determineswhether a apneic interval longer than or equal to a predetermined periodis included in this reference range (S208). In general, duringswallowing, respiration stops. In S208, it is determined whetherswallowing has occurred in the biological sound generation interval ofthe reference destination, from the viewpoint of respiration. In thedetermination in S208, the predetermined period is set to be longer thanor equal to 400 msec, for example; and preferably, set to be longer thanor equal to 500 msec, or longer than or equal to 600 msec. However, thepredetermined period is not limited to these, and can be set to anothervalue as long as the value can set the lower limit of the period inwhich respiration stops during swallowing.

Further, the control part 313 determines whether at least one of thepulses (pulses obtained in S205) that correspond to the biological soundgeneration interval of the reference destination is included in theapneic interval (S209). Here, it is determined whether sound has beendetected in the apneic interval. That is, whether sound has beendetected while respiration has stopped is used as a further swallowingestimation condition. When having determined as YES in both S208 andS209, the control part 313 estimates that the sound in the biologicalsound generation interval is swallowing sound, and specifies thisbiological sound generation interval as a swallowing sound generationinterval (S210). On the other hand, when having determined as NO ineither S208 or S209, the control part 313 determines that swallowing hasnot occurred in the biological sound generation interval (S211).

Subsequently, the control part 313 determines whether the processes ofS208 to S211 have ended for all the biological sound generationintervals extracted in S206 (S212). When the processes of S208 to S211have not ended (S212: NO), the control part 313 sets the next biologicalsound generation interval as the reference destination (S213) andreturns the process to S208. In this manner, the processes of S208 toS211 are performed for all the biological sound generation intervalsextracted in S206, whereby swallowing estimation is performed.

With reference to FIG. 8, next, the control part 313 sequentially refersto the swallowing sound generation intervals extracted in S210, anddetermines whether there is an aspiration risk in the swallowing soundgeneration interval concerned.

First, the control part 313 sets the first swallowing sound generationinterval as a reference destination (S214). Subsequently, the controlpart 313 obtains respiratory phases immediately before and immediatelyafter this swallowing sound generation interval (S215). Subsequently,the control part 313 determines whether the respiratory phaseimmediately before this swallowing sound generation interval is aninspiratory phase (S216), and further determines whether the respiratoryphase immediately after this swallowing sound generation interval is aninspiratory phase (S217). When having determined as YES in either S216or S217, the control part 313 determines that there is an aspirationrisk in the swallowing sound generation interval (S218). On the otherhand, when having determined as NO in both S216 and S217, the controlpart 313 determines that there is no aspiration risk in the swallowingsound generation interval (S219).

Subsequently, the control part 313 determines whether the processes ofS215 to S219 have ended for all the swallowing sound generationintervals (S220). When the processes of S215 to S219 have not ended(S220: NO), the control part 313 sets the next swallowing soundgeneration interval as the reference destination (S221) and returns theprocess to S215. In this manner, with respect to all the swallowingsound generation intervals, whether there is an aspiration risk isdetermined.

Next, in response to an instruction from the operator inputted via theinput part 330, the control part 313 performs a process of displaying,on the display part 320, a screen 410 (see FIG. 11), a screen 420 (seeFIG. 12), and a screen 430 (see FIG. 13A) based on the above processes(S222). Then, the process performed by the information processing device300 ends.

FIG. 11 shows the screen 410 displayed on the display part 320. Thescreen 410 includes icons 401, a reference position operation part 402,and a zoom-in/zoom-out operation part 403. In the screen 410, graphs ofanalog waveforms of the biological sound data, the airflow pressuredata, and the hyoid bone displacement data, and a graph of biologicalsound signal waveform intensity are shown.

On the graph of the biological sound data, broken lines are shown atpositions that correspond to the swallowing sound generation intervals,and the icons 401 are disposed above the broken lines, respectively.When an icon 401 is pressed, the screen 420 showing a zoomed-in state ofthe corresponding swallowing sound generation interval (see FIG. 12) isdisplayed on the display part 320. When the reference position operationpart 402 is operated, the time ranges of the respective pieces of datadisplayed on the four graphs are moved in a direction of advancing orreversing the time in the entire measurement interval. When thezoom-in/zoom-out operation part 403 is operated, the time widths of therespective pieces of data displayed on the four graphs areextended/compressed.

FIG. 12 shows the screen 420 to be displayed on the display part 320.The screen 420 includes an icon 421, a play button 422, a stop button423, a reproduction position operation part 424, and a close button 425.

In the screen 420, the analog waveform of the biological sound data andthe analog waveform of the airflow pressure data are shown in asuperposed manner. Here, the analog waveform of the biological sounddata is indicated with a solid line, and the analog waveform of theairflow pressure data is indicated with a dotted line. An interval S1 isa swallowing sound generation interval designated by the icon 401 in thescreen 410 of FIG. 11, and the icon 421 indicates this interval. Aninterval S2 is an apneic interval in the vicinity of the interval S1.

When the play button 422 is pressed, sound that is obtained byreproducing the biological sound data of the interval S1 is outputtedfrom the speaker 312. When the stop button 423 is pressed, thereproduction is stopped. The reproduction position operation part 424indicates the position of the interval S1 of the sound to be reproducedin the entire measurement interval. When the reproduction positionoperation part 424 is operated, the position at which the reproductionis performed is changed in a direction of advancing or reversing thetime.

FIG. 13A shows the screen 430 to be displayed on the display part 320.

The screen 430 shows: “the number of swallows” which indicates thenumber of swallowing sound generation intervals extracted in the entiremeasurement interval of the biological sound data; “the number ofinspirations-swallows” which indicates the number of swallowing soundgeneration intervals that each have an inspiratory phase immediatelytherebefore; “the number of swallows-inspirations” which indicates thenumber of swallowing sound generation intervals that each have aninspiratory phase immediately thereafter; and “the number of aspirationrisks” which indicates the number of swallowing sound generationintervals in which it has been determined that there had been anaspiration risk in S218 shown in FIG. 8.

As described above, according to Embodiment 2, whether swallowing hasoccurred is estimated based on the biological sound in an apneic state,and thus, the swallowing estimation accuracy can be increased.Accordingly, even if information for swallowing estimation is collectedin a living environment, high swallowing estimation accuracy can bemaintained. As a swallowing determination condition, values ofparameters are used which indicate intermittency and continuity ofsound, that is, the number and the lengths of pulses obtained byperforming short-time Fourier transform and wavelet transform on thebiological sound data. Thus, swallowing estimation can be performed withhigh accuracy through calculation. It should be noted that, among thebiological sound generation intervals, only biological sound generationintervals that each correspond to an apneic interval are targeted forcalculation. Accordingly, the calculation load is reduced, andswallowing estimation can be efficiently performed.

Moreover, biological sound generation intervals that each correspond toan apneic interval longer than or equal to 400 msec are targeted forswallowing estimation, whereby the swallowing estimation accuracy isfurther increased. Normally, during swallowing, respiration is stoppedfor a relatively long time. Thus, by causing the length of thenon-respiration interval to be included in the estimation condition, theswallowing estimation accuracy is further increased.

As shown in S209, using as a further estimation condition whether atleast one pulse is included in the apneic interval, it is estimatedwhether swallowing has occurred in the biological sound generationinterval targeted for determination. Thus, since whether sound has beendetected in a period where respiration has stopped is used as a furtherestimation condition, a high accuracy swallowing estimation result canbe obtained.

Further, as shown in S206, in each biological sound generation interval,the proportion of sound having a frequency band higher than or equal to750 Hz is calculated. By using as a further estimation condition whetherthe calculated proportion exceeds a predetermined proportion, it isestimated whether swallowing has occurred in the biological soundgeneration interval. Since the frequency components of sound are used asa further estimation condition, a high accuracy swallowing estimationresult can be obtained.

As shown in S208, using as a further estimation condition whether thelength of the apneic interval included in the biological soundgeneration interval exceeds a threshold, it is estimated whetherswallowing has occurred in the biological sound generation interval.Since the length of the apneic interval included in the biological soundgeneration interval is used as a further estimation condition, a highaccuracy swallowing estimation result can be obtained.

As shown in S206, using as a further estimation condition whether thebiological sound generation interval includes an amplitude of the hyoidbone displacement data whose proportion to the maximum amplitude thereofin the entire interval is greater than or equal to a predeterminedproportion, it is estimated whether swallowing has occurred in thebiological sound generation interval. Since the amount of displacementof the larynx portion is used as a further estimation condition, a highaccuracy swallowing estimation result can be obtained.

When the play button 422 is pressed in the screen 420, the biologicalsound data in the designated swallowing sound generation interval isreproduced. Thus, by actually listening to the sound at the timing atwhich it has been estimated that swallowing had occurred, a doctor orthe like can confirm whether swallowing actually occurred at the timing.It should be noted that the reproduction interval of the biologicalsound data is not limited to the designated swallowing sound generationinterval, and may be an interval having a time width previously setbefore and/or after the center of designated swallowing sound generationinterval, for example. Alternatively, an interval obtained by adding apredetermined time width before and/or after the designated swallowingsound generation interval may be used as the reproduction interval ofthe biological sound data.

Further, in the screen 410, the icons 401 are disposed at positions thatcorrespond to the swallowing sound generation intervals, respectively,and when an icon 401 is pressed, the screen 420 is displayed. Thus, thedoctor or the like can easily designate the timing for which he or shewould like to confirm, through sound, whether swallowing occurred.

In each of the screens 410 and 420, a graph of the analog waveform ofthe biological sound data is shown. Thus, the doctor or the like canappropriately designate the timing for which he or she would like toconfirm, through sound, whether swallowing occurred, while confirmingthe sound waveform by viewing it.

When the icon 401 is pressed on the screen 410, a correspondingswallowing sound generation interval is displayed in the screen 420 in azoomed-in manner. When the zoom-in/zoom-out operation part 403 isoperated on the screen 410, the data displayed on the graph in thescreen 410 can be zoomed-in. Thus, while confirming the sound waveformby viewing it, the doctor or the like can listen to the sound. Thisenables more appropriate determination of whether swallowing actuallyoccurred at the timing.

As shown in FIG. 8, the respiratory phases before and after the timingat which it has been estimated that swallowing had occurred aredetected; and based on the detected respiratory phases, it is evaluatedwhether there has been a possibility of aspiration at the timing. Then,the result of the evaluation is displayed in the screen 430. Thus, thedoctor or the like can know whether it has been determined that therehad been an aspiration risk for the subject by viewing the display, andthus, can utilize this knowledge in diagnosis for the subject.

<Modification 1>

In Embodiment 2, swallowing sound generation intervals are extracted byusing three types of data, i.e., the biological sound data, the airflowpressure data, and the hyoid bone displacement data. However, inModification 1, swallowing sound generation intervals are extracted byusing the biological sound data and the airflow pressure data among thethree types of data above.

In Modification 1, as shown in FIG. 13B, in the process performed by theinformation processing device 300 shown in FIG. 6, S206 is replaced withS301. In S301, different from Embodiment 2, the first condition usingthe hyoid bone displacement data is omitted.

FIG. 13C shows estimation results obtained when swallowing estimationwas performed by actually collecting information for swallowingestimation.

Here, as shown in FIG. 4, the nasal cannula 220 and the pad 231 wereattached to the subject, and information was collected in a livingenvironment. The subject performs various actions spontaneously (in apseudo manner) such as rotating the neck, coughing, uttering sounds,swallowing air, burping, sniffling, snoring, and deep breathing, inaddition to swallowing. Then, every time performing such an action, thesubject wrote down the time and the content of the action. In addition,in a room adjacent to the room of the subject, utterance and the likewere made to generate household noises. Also with respect to such anoise, the time and the content were written down. Actions by thesubject and generation of household noises were made 87 times in total.Among them, swallowing was conducted 27 times.

In FIG. 13C, swallowing estimation using “3 elements (Embodiment 2)” wasperformed in the following steps. In the following steps, thepredetermined period was set to 600 msec.

(1) From the biological sound data, biological sound generationintervals are extracted by the method described above.

(2) In each biological sound generation interval, an apneic intervallonger than or equal to the predetermined period is detected. Biologicalsound generation intervals in which an apneic interval longer than orequal to the predetermined period is not detected are not targeted forswallowing estimation.

(3) In each apneic interval longer than or equal to the predeterminedperiod, the biological sound data is converted into pulses.

(4) With respect to each apneic interval longer than or equal to thepredetermined period, the determination (using three parameters of hyoidbone displacement, biological sound frequency, and biological soundpulse) in S206 in FIG. 6 is conducted, and whether the biological soundgeneration interval is an estimated swallowing interval is determined.

It should be noted that the order of step (2) and step (3) may beswitched with each other. In this case, conversion of the biologicalsound into pulses is performed on all the biological sound generationintervals. Then, an apneic interval longer than or equal to thepredetermined period is detected for each biological sound generationinterval. Then, whether the biological sound generation interval is anestimated swallowing interval is determined based on the widths and thenumber of the sound pulses in the apneic interval.

In FIG. 13C, with respect to “without hyoid bone (Modification 1)”, instep (4) of the above algorithm, the parameter regarding displacement ofthe hyoid bone was omitted from the parameters used in thedetermination. With respect to “without sound (Comparative Example 1)”,in step (4) of the above algorithm, the parameter based on thebiological sound was omitted from the parameters used in thedetermination. Further, with respect to “without respiration(Comparative Example 2)”, in the above algorithm, step (2) was omitted,the biological sound data was converted into pulses for all thebiological sound generation intervals in step (3), and the determinationin S206 in FIG. 6 was performed for all the biological sound generationintervals in step (4), whereby it was determined whether the biologicalsound generation interval concerned was an estimated swallowinginterval.

The lines in FIG. 13C indicate, from the top: the number of swallowsthat were performed by the subject and that could be extracted throughthe estimation process; the number of swallows that were performed bythe subject and that could not be extracted through the estimationprocess (non-extractions); the number of swallows that were not actuallyswallows but were determined as swallows through the estimation process(erroneous extractions); and the total thereof. The rows are, from left,estimation results obtained in Embodiment 2, Comparative Example 1,Comparative Example 2, and Modification 1.

With reference to FIG. 13C, in each of the estimation processes, all 27swallows performed by the subject were correctly estimated as swallows.However, in Comparative Example where the airflow pressure (respiration)was not used as a parameter, actions other than swallowing and householdnoises were estimated as swallows as many as 36 times. This numbergreatly exceeds 27 which is the number of correct swallowingestimations. In contrast, in Embodiment 2, Comparative Example 1, andModification 1 where the airflow pressure (respiration) was used as aparameter, occurrence of erroneous swallowing extraction was suppressed.In particular, in Embodiment 2 where three parameters including theairflow pressure (respiration) were used, the number of erroneousswallowing extractions was 7, which was a greatly reduced number.

The estimation results shown in FIG. 13C reveal that by causing theparameter of respiration to be included as a swallowing estimationcondition, the rate of erroneous swallowing extraction is greatlyreduced. In particular, as in Embodiment 2 above, by causing theparameter of biological sound and the parameter of hyoid bonedisplacement to be included in the swallowing estimation conditions aswell as the parameter of respiration, occurrence of erroneous swallowingextraction can be remarkably suppressed and swallowing estimation can beperformed with high accuracy.

With respect to the estimation results shown in FIG. 13C, in theestimation process in Embodiment 2, coughs by the subject were extractedas swallows. Since almost all the sound of a cough occurs duringexpiration, the sound of a cough is excluded in step (3), andtheoretically, the cough is not determined as a swallow in normalsituations. However, actually, coughs were estimated as swallows in themeasurement. The reason for this is assumed as follows: in themeasurement in FIG. 13C, each apneic interval was extracted inconsideration of noise; thus, a small (greater than or equal to thepredetermined period in steps (3) and (4)) expiration interval isincluded in the apneic interval; and due to the sound of a cough and themovement of the hyoid bone in this expiration interval, the cough wasestimated as a swallow. Occurrence of such an erroneous extraction canbe suppressed by further setting a condition “in the extractedbiological sound interval, sound in the apneic interval is louder thansound in the respiration” in addition to the steps (1) to (4) above.This is because the sound of a cough at the start of expiration includedin an apneic interval is louder in the expiration interval thereafter.

<Modification 2>

In Modification 2, among biological sound generation intervals, onlybiological sound generation intervals that each correspond to an apneicinterval longer than or equal to a predetermined period are targeted forswallowing estimation.

FIG. 14 is a flow chart showing the process performed in this case. InFIG. 14, S203 to S209 in the flow charts in FIGS. 6 and 7 are replacedwith S311 to S315.

When biological sound generation intervals have been extracted from thebiological sound data in S202, biological sound generation intervals,among the extracted biological sound generation intervals, that eachinclude an apneic interval longer than or equal to a predeterminedperiod are set as reference targets for swallowing estimation (S311).Among the biological sound generation intervals set as the referencetargets, the first biological sound generation interval is referred to(S312), and the biological sound data included in the apneic interval inthe biological sound generation interval is converted into pulses as inEmbodiment 2 above (S313). Then, the swallowing condition in S206 inFIG. 6 is applied to the value of each parameter for this apneicinterval (S314), and whether the values of all the parameters satisfythe swallowing condition is determined (S315). When the values of allthe parameters satisfy the swallowing condition (S315: YES), it isdetermined that swallowing has occurred in the biological soundgeneration interval (S210). When the value of at least one parameterdoes not satisfy the swallowing condition (S315: NO), it is determinedthat swallowing has not occurred in the biological sound generationinterval (S211).

In Modification 2, among biological sound generation intervals, onlybiological sound generation intervals that each include an apneicinterval longer than or equal to a predetermined period are targeted forswallowing estimation. Accordingly, calculation load can be reduced, andswallowing estimation can be more efficiently performed.

<Modification 3>

In Modification 3, the control part 313 compares biological sound datain the respiratory phase immediately before each swallowing soundgeneration interval with biological sound data in the respiratory phaseimmediately after the swallowing sound generation interval, therebydetermining whether aspiration has occurred in the swallowing soundgeneration interval.

With reference to the left part of FIG. 15, the control part 313performs Fourier transform on biological sound data in the inspiratoryphase immediately before a swallowing sound generation interval andtakes the integral from frequency F1 to frequency F2, therebycalculating a value A. Further, the control part 313 performs Fouriertransform on biological sound data in the expiratory phase immediatelybefore the swallowing sound generation interval and takes the integralfrom frequency F1 to frequency F2, thereby calculating a value B.Similarly, the control part 313 performs Fourier transform on biologicalsound data in the inspiratory phase immediately after the swallowingsound generation interval and takes the integral from frequency F1 tofrequency F2, thereby calculating a value A′. Further, the control part313 performs Fourier transform on biological sound data in theexpiratory phase immediately after the swallowing sound generationinterval and takes the integral from frequency F1 to frequency F2,thereby calculating a value B′.

Next, the control part 313 calculates a power ratio A/B based on thevalues obtained for the respiratory phases immediately before theswallowing sound generation interval, and a power ratio A′/B′ based onthe values obtained for the respiratory phases immediately after theswallowing sound generation interval. Then, when the power ratio A′/B′has increased by a predetermined amount (for example, 50%) from thepower ratio A/B, the control part 313 determines that aspiration hasoccurred in the swallowing sound generation interval. In this manner,whether aspiration has occurred is determined with respect to all theswallowing sound generation intervals. Similarly to Embodiment 2, thenumber of aspiration risks in this case is also displayed in the screen430 as shown in FIG. 13A.

According to Modification 3, inspiration sound and expiration soundbefore and after the timing at which it has been estimated thatswallowing had occurred are detected. Then, based on the detectedinspiration sound and expiration sound, it is evaluated whether therehas been an aspiration risk at that timing. Then, the result of theevaluation is displayed in the screen 430. Thus, similarly to Embodiment2, the doctor or the like can know whether it has been determined thatthere had been an aspiration risk for the subject by viewing thedisplay, and thus, can utilize this knowledge in diagnosis for thesubject.

<Other Modifications>

In S202 of Embodiment 2 and Modification 2, short-time Fourier transformis performed on biological sound data, then, the total sum of theamplitudes is calculated to obtain signal intensity, and then, theobtained signal intensity is compared with a threshold, wherebybiological sound generation intervals are extracted. However, the methodfor extracting biological sound generation intervals is not limitedthereto. For example, biological sound is subjected to full-waverectification, then, to leak integration, and an interval that has avalue exceeding the average+2SD (standard deviation) of the value in thesoundless interval thereof may be defined as a biological soundgeneration interval. Also, the method for setting the threshold is notlimited thereto, and as long as a biological sound generation intervalcan be extracted, another method may be used.

In Embodiment 2, various types of data is sent from the terminal device210 to the information processing device 300 by means of the storagemedium 101. However, the sending method is not limited thereto, andvarious types of data may be transmitted from the storage part of theterminal device 210 to the information processing device 300 over awired or wireless communication network. Further, the terminal device210 may have the function of the information processing device 300 andthe information processing device 300 may be omitted. In this case, thecontrol part 214 of the terminal device 210 performs all the processesperformed by the information processing device shown in FIGS. 6 to 8,and outputs screens and the like showing results to the display part211.

In addition to the above, various modifications can be made asappropriate, without departing from the scope of the technical ideadefined by the claims.

From the embodiments above, an invention according to the claim belowcan also be derived. In this invention, parameters used in swallowingestimation are not limited to the parameters presented in theembodiments above. One or more of the parameters above can be combinedtogether. Alternatively, other parameters can also be used. This claimcan have claims 8 and 9 dependent therefrom.

<Claim>

A swallowing estimation device comprising:

a sound detection part configured to detect sound of a larynx portion;

a storage part in which sound information outputted from the sounddetection part is stored;

a swallowing estimation part configured to estimate swallowing;

an output part configured to output information based on a result of theestimation performed by the swallowing estimation part; and

an input part capable of designating a timing at which the swallowingestimation part has estimated that swallowing had occurred, wherein

the output part obtains from the storage part the sound informationhaving a time width including the timing designated via the input part,and outputs, to outside, sound that is obtained by reproducing theobtained sound information.

According to this invention, a significant effect can be obtained thatthe doctor or the like actually listens to the sound at the timing atwhich it has been estimated that swallowing had occurred, thereby beingable to confirm whether swallowing actually occurred at the timing.

Moreover, from the embodiments above, an invention according to theclaim below can also be derived. This claim can be dependent from claim8 or 9.

<Claim>

A swallowing estimation device wherein

the output part obtains from the storage part the sound informationhaving a time width including the timing designated via the input part,and displays a sound waveform based on the obtained sound information,in a zoomed-in manner.

According to this invention, the doctor or the like can listen to thesound, while confirming the sound waveform by viewing it. Accordingly,it is possible to more appropriately determine whether swallowingactually occurred at the timing.

INDUSTRIAL APPLICABILITY

The swallowing estimation device according to the present invention hasan excellent swallowing estimation function, and can be used in thefield of medical devices.

What is claimed is:
 1. A swallowing estimation device comprising: asound detection part configured to detect sound of a larynx portion; arespiration detection part configured to detect respiration; and aswallowing estimation part configured to estimate swallowing based onsound information outputted from the sound detection part and based onrespiration information outputted from the respiration detection part,wherein the swallowing estimation part obtains a value of a parameterfor swallowing estimation with respect to a biological sound generationinterval that corresponds to a respiratory cessation (apnea) intervallonger than or equal to 400 msec, and estimates whether swallowing hasoccurred in the biological sound generation interval based on whetherthe obtained value of the parameter satisfies a swallowing determinationcondition.
 2. The swallowing estimation device according to claim 1,wherein as the value of the parameter for the swallowing estimation, theswallowing estimation part obtains, through calculation, a value of aparameter indicating intermittency and continuity of the sound.
 3. Theswallowing estimation device according to claim 2, wherein theswallowing estimation part obtains a pulse signal by performingcalculation processing of Fourier transform and wavelet transform on thesound information, and obtains the number and a length of the obtainedpulse signal as the value of the parameter.
 4. The swallowing estimationdevice according to claim 1, wherein the swallowing estimation partcalculates a proportion of sound having a frequency band exceeding apredetermined frequency in the biological sound generation interval, andestimates whether swallowing has occurred in the biological soundgeneration interval, using as a further estimation condition whether thecalculated proportion exceeds a threshold.
 5. The swallowing estimationdevice according to claim 1, further comprising a displacement detectionpart configured to detect displacement of the larynx portion, whereinusing as a further estimation condition whether an amount of thedisplacement of the larynx portion detected by the displacementdetection part in the biological sound generation interval exceeds athreshold, the swallowing estimation part estimates whether swallowinghas occurred in the biological sound generation interval.
 6. Theswallowing estimation device according to claim 1, comprising an outputpart configured to output information based on a result of theestimation performed by the swallowing estimation part.
 7. Theswallowing estimation device according to claim 6, comprising: a storagepart in which the sound information outputted from the sound detectionpart is stored; and an input part capable of designating a timing atwhich the swallowing estimation part has estimated that swallowing hadoccurred, wherein the output part obtains from the storage part thesound information having a time width including the timing designatedvia the input part, and outputs, to outside, sound that is obtained byreproducing the obtained sound information.
 8. The swallowing estimationdevice according to claim 7, wherein the output part displays a screenin which the timing at which it has been estimated that swallowing hadoccurred is superposed on a time axis, and the input part is configuredto be capable of designating the timing displayed in the screen.
 9. Theswallowing estimation device according to claim 8, wherein the outputpart displays, along with the timing, a sound waveform based on thesound information so as to be superposed on the time axis.
 10. Theswallowing estimation device according to claim 7, wherein theswallowing estimation part detects, from the respiration information,respiratory phases before and after the timing at which the swallowingestimation part has estimated that swallowing had occurred, andevaluates whether there has been a possibility of aspiration at thetiming based on the detected respiratory phases, and the output partoutputs information based on a result of the evaluation performed by theswallowing estimation part.
 11. The swallowing estimation deviceaccording to claim 7, wherein the swallowing estimation part detects,from the sound information, inspiration sound and expiration soundbefore and after the timing at which the swallowing estimation part hasestimated that swallowing had occurred, and evaluates whether there hasbeen a possibility of aspiration at the timing based on the detectedinspiration sound and expiration sound, and the output part outputsinformation based on a result of the evaluation performed by theswallowing estimation part.
 12. A swallowing estimation devicecomprising: a biological sound detection means configured to detectbiological sound at a larynx portion; respiration detection meansconfigured to detect change in airflow of respiration; a signalintensity conversion means configured to convert biological sound dataobtained by sampling the biological sound into signal intensity data;signal interval identification means configured to identify a signalinterval having an intensity level higher than or equal to a noise levelbased on the signal intensity data; respiration identification meansconfigured to identify an apneic interval based on airflow pressure dataobtained by sampling change in the respiration; signal pulsing meansconfigured to obtain a signal intensity that corresponds to a samplingtiming in the apneic interval that is longer than or equal to apredetermined period and that overlaps the signal interval, andconfigured to generate a signal pulse having a width that corresponds toa period in which the signal intensity is greater than or equal to apredetermined level; swallowing reflex estimation means configured toestimate, as an estimated swallowing reflex interval, the apneicinterval that satisfies a determination condition that the number of thesignal pulses in the apneic interval longer than or equal to thepredetermined period is less than or equal to a predetermined number anda width of each signal pulse in the apneic interval longer than or equalto the predetermined period is less than or equal to a predeterminedperiod; and display means configured to display the estimated swallowingreflex interval.
 13. An information terminal device comprising: a sounddetection part configured to detect sound of a larynx portion; arespiration detection part configured to detect respiration; and astorage part in which sound information outputted from the sounddetection part and respiration information outputted from therespiration detection part are stored.
 14. A storage medium havingstored therein a program which provides a computer with: a function ofobtaining a value of a parameter for swallowing estimation, with respectto a biological sound generation interval that corresponds to an apneicinterval longer than or equal to a predetermined period; and a functionof estimating whether swallowing has occurred in the biological soundgeneration interval based on whether the obtained value of the parametersatisfies a swallowing determination condition.